As computer systems have advanced, processing power and speed have increased substantially. Computer systems have thus been able to solve increasing complicated problems. Iterative methods for solving large sparse linear systems of equations have become popular in many areas of scientific computing. Although direct solution methods have long been preferred because of their robustness and predictable behavior, the development of new efficient iterative solvers and the increased need for solving very large systems have caused iterative solvers to become the method of choice for solving sparse linear systems.
A wide variety of iterative algorithms exist to solve sparse linear systems of equations including stationary iterative methods (such as Jacobi, Gauss-Seidel, Successive Over-Relaxation (SOR)), Krylov subspace methods (such as Conjugate Gradient (CG), Bi-Conjugate Gradient (BiCG), Generalized Minimal Residual Method (GMRES)) and Algebraic MultiGrid (AMG) methods. Krylov subspace methods and AMG methods have been the most popular iterative methods to solve sparse linear systems arising from partial differential equations (PDEs) because of their robustness and efficiency. Unfortunately, these iterative algorithms can be sequential in nature. This sequential nature results from the dependencies between computations and thereby results in increased computation time as each computation is dependent upon the needs results from proceeding computations.
Thus, while iterative algorithms are desirable over direct solving methods because of their efficiency, the sequential nature of the computations limits performance and time saved.